Cross-environment application of tracing information for improved code execution

ABSTRACT

Systems and methods are described for enabling cross-environment application of tracing information for code, such as code executed within an on-demand (or “serverless”) code execution system. Various optimizations exist that allow execution of code to proceed faster or more efficiently over time, by collecting tracing information regarding the execution and using that tracing information to guide compilation of the code. These optimizations are typically designed for long-lived environments. However, executions within an on-demand code execution system often occur in short-lived environments, reducing or eliminating any gains from these optimizations. To address this issue, optimizations made in a first environment based on tracing information can be passed to a subsequent environment, enabling those optimizations to persist across short-lived environments.

BACKGROUND

Computing devices can utilize communication networks to exchange data.Companies and organizations operate computer networks that interconnecta number of computing devices to support operations or to provideservices to third parties. The computing systems can be located in asingle geographic location or located in multiple, distinct geographiclocations (e.g., interconnected via private or public communicationnetworks). Specifically, data centers or data processing centers, hereingenerally referred to as a “data center,” may include a number ofinterconnected computing systems to provide computing resources to usersof the data center. The data centers may be private data centersoperated on behalf of an organization or public data centers operated onbehalf, or for the benefit of, the general public.

To facilitate increased utilization of data center resources,virtualization technologies allow a single physical computing device tohost one or more instances of virtual machines that appear and operateas independent computing devices to users of a data center. Withvirtualization, the single physical computing device can create,maintain, delete, or otherwise manage virtual machines in a dynamicmanner. In turn, users can request computer resources from a datacenter, including single computing devices or a configuration ofnetworked computing devices, and be provided with varying numbers ofvirtual machine resources.

In some scenarios, virtual machine instances may be configured accordingto a number of virtual machine instance types to provide specificfunctionality. For example, various computing devices may be associatedwith different combinations of operating systems or operating systemconfigurations, virtualized hardware resources and software applicationsto enable a computing device to provide different desiredfunctionalities, or to provide similar functionalities more efficiently.These virtual machine instance type configurations are often containedwithin a device image, which includes static data containing thesoftware (e.g., the OS and applications together with theirconfiguration and data files, etc.) that the virtual machine will runonce started. The device image is typically stored on the disk used tocreate or initialize the instance. Thus, a computing device may processthe device image in order to implement the desired softwareconfiguration.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram depicting an illustrative environment in whichan on-demand code execution system can operate to execute taskscorresponding to code, which may be submitted by users of the on-demandcode execution system, and to enable cross-environment application oftracing information to improve execution of that code;

FIG. 2 depicts a general architecture of a computing device providing anacceleration system that is configured to facilitate cross-environmentapplication of tracing information for code of tasks on the on-demandcode execution system of FIG. 1;

FIG. 3 is a flow diagram depicting illustrative interactions forcollecting trace information during execution of source code within afirst execution environment and submitting that trace information to theacceleration system of FIG. 1, which may use that trace information toconduct assistant ahead-of-time (AOT) compilation for the source code;

FIG. 4 is a flow diagram depicting illustrative interactions forgenerating machine code based on application of a just-in-time (JIT)compiler to source code executing within a first execution environmentand submitting that machine code to the acceleration system of FIG. 1;

FIG. 5 is a flow diagram depicting the use of optimization information,such as tracing information or machine code collected with respect toexecution of source code in a first execution environment, to improvecode execution in a second execution environment; and

FIG. 6 is a flow chart depicting an illustrative routine for enablingcross-environment application of tracing information for code of taskson the on-demand code execution system of FIG. 1.

DETAILED DESCRIPTION

Generally described, aspects of the present disclosure relate to anon-demand code execution system, which may also be referred to as a“serverless” execution system. The on-demand code execution systemenables rapid execution of source code, which may be supplied by usersof the on-demand code execution system. The on-demand code executionsystem facilitates such execution, for example, by interpreting orcompiling the source code, and executing a resulting representation ofthe code (often machine code) within an execution environment. Theon-demand code execution system may also facilitate generation of thatenvironment, and deletion of that environment on completing execution ofthe source code. In one embodiment, execution of source code occursrapidly (e.g., in under 500 milliseconds (ms)), and thus the lifetime ofan execution environment may be comparatively short. While this mayprovide benefits to the on-demand code execution system, such as areduction in computing resources required to execute source code, theshort lifetime of an execution environment may also prevent or inhibitexecution optimization techniques originally conceived for long-runningenvironments. One such optimization technique is tracing-basedjust-in-time (JIT) compilation. This technique, which is generally knownin the art, utilizes trace information collected during execution of aprogram to guide future JIT compilation for the program. This techniquecan greatly increase the speed of execution of source code, by reducingthe need to interpret that source code during execution or by enablingother compilation optimizations. However, traditional tracing-based JITcompilation operates solely within a given execution environment, andtypically during a single execution of source code. Moreover,traditional tracing-based JIT compilation generally only providesbenefits after a sufficient amount of tracing information is collected,which can often take significant amounts of time (e.g., thousands ofexecutions of a given portion of code). This time is sometimes referredto as “JIT warmup” time. During JIT warmup, execution of code canactually be slowed relatively to a device not utilizing tracing-basedJIT compilation, as such compilation requires additional computingresources to conduct tracing and optimization. For this reason, thebenefits of tracing-based JIT compilation are often realized only inlong-running environments, and use of this technique in short-livedenvironments can actually be detrimental. The rapid construction anddeconstruction of execution environments, such as might occur within theon-demand code execution system, can thus prevent gains in efficiencyfrom tracing-based JIT compilation. The present disclosure addressesthis problem in traditional tracing-based JIT compilation by enablingcross-environment application of tracing information, thus enabling thebenefits of traditional tracing-based JIT compilation within short-livedexecution environments.

Specifically, and as will be described in more detail below, aspects ofthe present disclosure enable tracing information gathered duringexecution of source code in a first execution environment on anon-demand code execution system to be used to more efficiently executethe code in a second execution environment. In one embodiment, a deviceimplementing a first environment may operate, during execution of sourcecode, to conduct tracing of the execution. Tracing may occur, forexample, due to operation of a tracing JIT compiler within the firstenvironment, in accordance with traditional tracing-based JIToptimization. However, unlike traditional tracing-based JIToptimization, the tracing information may be made available to a secondexecution environment. For example, the device implementing the firstenvironment may submit the tracing information to an accelerationservice of the on-demand code execution system. The acceleration servicemay then make the tracing information collected during operation of thefirst execution environment available to a second execution environmentexecuting the same source code. Illustratively, the acceleration servicemay cause the tracing information to be placed into memory accessible tothe second execution environment (e.g., persistent memory such as ahard-disk drive or non-persistent memory such as random access memory(RAM)). A JIT compiler operating within the second execution environmentmay then reference the tracing information of the first executionenvironment as if that information were created within the secondexecution environment. In this way, operation of the JIT compiler withinthe second execution environment may be “jump started” with the tracinginformation of the first execution environment, enabling the JITcompiler of the second execution environment to reach a stage of codeoptimization more quickly than if the JIT compiler of the secondexecution environment were itself to generate all tracing informationused for code optimization. Thus, by transmission of tracing informationbetween execution environments, tracing-based JIT compilation can beenabled even for short-lived environments.

Additionally or alternatively to sharing of tracing information betweenenvironments, the on-demand code execution system of the presentdisclosure may utilize tracing information gathered during execution ofsource code within a first environment to implement other codeoptimization techniques for source code to be executed in a secondexecution environment. Illustratively, the acceleration service of theon-demand code execution system may utilize tracing information gatheredduring execution of source code within a first environment to conductguided ahead-of-time (AOT) compilation of the source code. Generally,AOT compilation stands in contract to JIT compilation, in that it occursprior to execution of the source code. This technique may reduce thecomputing resources needed to execute code, as some portion ofcompilation for the code has already occurred. AOT compilation generallyoccurs without reference to information gathered during execution ofcode (generally referred to as “runtime” information), thus forgoingmany of the beneficial optimizations that occur in JIT compilation.However, in accordance with aspects of the present disclosure, anacceleration service may conduct AOT compilation for source code basedon tracing information gathered during a past execution of that sourcecode within an execution environment. This technique may thus enable AOTcompilation to integrate optimizations typically possible only duringJIT compilation, combining the benefits of AOT and JIT compilation.Machine code generated by the acceleration service can then be providedto subsequent execution environments, thus reducing the need for JIToptimizations within the subsequent execution environments and speedingexecution of the code.

In yet another embodiment of the present disclosure, in addition or asan alternative to transfer of tracing information between environments,the efficiency of execution of code within a second environment can beincreased by transfer of machine code generated within a first executionenvironment to the second environment. Illustratively, a JIT compilerwithin a first execution environment may operate, during execution ofsource code, to generate machine code for one or more portions of thesource code (e.g., as guided by tracing information gathered within thefirst execution environment). In accordance with traditionaltracing-based JIT optimization, machine code may allow the source codeto execute more quickly, by reducing a need to interpret portions of thesource code (which portions can instead be directly executed in theirmachine code representation). However, in traditional tracing-based JIToptimization, machine code generated during execution of source code isgenerally machine-dependent, and as such is not used in execution of thesource code within other execution environments. In accordance with thepresent disclosure, this machine code generated within a firstenvironment may be transmitted to an acceleration service, which may inturn provision subsequent execution environments with the machine code.Thus, rather than being required to itself generate machine code for thesource code, a device implementing the second environment may directlyutilize the machine code of the first environment to execute the sourcecode. To ensure the validity of the machine code of the firstenvironment to the second environment, the on-demand code executionsystem may be configured to ensure that the second environment matchesan architecture of the first environment. For example, the on-demandcode execution system may ensure that the second environment operates ona common operating system and processor architecture (e.g., as a realphysical architecture or virtual architecture) with the firstenvironment.

Tracing JIT compilers exist for a variety of languages, and as suchsource code of the present disclosure may be expressed in a variety oflanguages. For example, source code of the present disclosure may beexpressed as Java, JavaScript, Python, or CIL language code. Examples ofthe present disclosure may be provided with respect to individuallanguages. It should be understood that such examples are provided forillustration only.

While aspects of the present disclosure may refer to non-machine codegenerally as “source code,” which may include human-readable computercode, it should be understood that the term source code, as used herein,may also refer to device-independent representation of source code, suchas an “intermediate representation” of the source code. Generallydescripted, an intermediate representation of source code can be a datastructure or code which accurately represents source code, but may bemore readily manipulated by a computing device (e.g., to conductoptimizations). Intermediate representations are generallydevice-independent, and represent instructions at a higher level thandevice-dependent code, such as machine code. Examples of intermediaterepresentations include Java bytecode or .NET Framework CommonIntermediate Language (CIL) code. Various techniques described herein,such as trace-based JIT compilation, may occur with respect to anintermediate representation of source code.

As will be appreciated by one of skill in the art in light of thepresent disclosure, the embodiments disclosed herein improves theability of computing systems, such as on-demand code execution systems,to execute source code. Specifically, aspects of the present disclosurereduce the computing resources required to execute source code withinmultiple execution environments, by utilizing information (e.g., tracinginformation or machine code generated based on tracing information,which may collectively be referred to herein as “optimizationinformation”) from a first execution environment to assist in executionof code within a second execution environment. Aspects of the presentdisclosure further speed the execute of code in the second environmentdue to the use of such optimization information between environments.Moreover, the presently disclosed embodiments address technical problemsinherent within computing systems; specifically, the limited resourcesof computers to execute code, and the transitive, device-specific natureof optimization information generated according to traditionaltechniques. These technical problems are addressed by the varioustechnical solutions described herein, including the utilization ofoptimization information (e.g., tracing information or machine codegenerated based on tracing information) across execution environments.Thus, the present disclosure represents an improvement on existing codeexecution systems and computing systems in general.

The general execution of tasks on the on-demand code execution systemwill now be discussed. As described in detail herein, the on-demand codeexecution system may provide a network-accessible service enabling usersto submit or designate computer-executable source code to be executed byvirtual machine instances on the on-demand code execution system. Eachset of code on the on-demand code execution system may define a “task,”and implement specific functionality corresponding to that task whenexecuted on a virtual machine instance of the on-demand code executionsystem. Individual implementations of the task on the on-demand codeexecution system may be referred to as an “execution” of the task (or a“task execution”). The on-demand code execution system can furtherenable users to trigger execution of a task based on a variety ofpotential events, such as detecting new data at a network-based storagesystem, transmission of an application programming interface (“API”)call to the on-demand code execution system, or transmission of aspecially formatted hypertext transport protocol (“HTTP”) packet to theon-demand code execution system. Thus, users may utilize the on-demandcode execution system to execute any specified executable code“on-demand,” without requiring configuration or maintenance of theunderlying hardware or infrastructure on which the code is executed.Further, the on-demand code execution system may be configured toexecute tasks in a rapid manner (e.g., in under 100 milliseconds [ms]),thus enabling execution of tasks in “real-time” (e.g., with little or noperceptible delay to an end user). To enable this rapid execution, theon-demand code execution system can include one or more virtual machineinstances that are “pre-warmed” or pre-initialized (e.g., booted into anoperating system and executing a complete or substantially completeruntime environment) and configured to enable execution of user-definedcode, such that the code may be rapidly executed in response to arequest to execute the code, without delay caused by initializing thevirtual machine instance. Thus, when an execution of a task istriggered, the code corresponding to that task can be executed within apre-initialized virtual machine in a very short amount of time.

Specifically, to execute tasks, the on-demand code execution systemdescribed herein may maintain a pool of pre-initialized virtual machineinstances that are ready for use as soon as a user request is received.Due to the pre-initialized nature of these virtual machines, delay(sometimes referred to as latency) associated with executing the usercode (e.g., instance and language runtime startup time) can besignificantly reduced, often to sub-100 millisecond levels.Illustratively, the on-demand code execution system may maintain a poolof virtual machine instances on one or more physical computing devices,where each virtual machine instance has one or more software components(e.g., operating systems, language runtimes, libraries, etc.) loadedthereon. When the on-demand code execution system receives a request toexecute the program code of a user (a “task”), which specifies one ormore computing constraints for executing the program code of the user,the on-demand code execution system may select a virtual machineinstance for executing the program code of the user based on the one ormore computing constraints specified by the request and cause theprogram code of the user to be executed on the selected virtual machineinstance. The program codes can be executed in isolated containers thatare created on the virtual machine instances. Since the virtual machineinstances in the pool have already been booted and loaded withparticular operating systems and language runtimes by the time therequests are received, the delay associated with finding computecapacity that can handle the requests (e.g., by executing the user codein one or more containers created on the virtual machine instances) issignificantly reduced.

The on-demand code execution system may include a virtual machineinstance manager configured to receive user code (threads, programs,etc., composed in any of a variety of programming languages) and executethe code in a highly scalable, low latency manner, without requiringuser configuration of a virtual machine instance. Specifically, thevirtual machine instance manager can, prior to receiving the user codeand prior to receiving any information from a user regarding anyparticular virtual machine instance configuration, create and configurevirtual machine instances according to a predetermined set ofconfigurations, each corresponding to any one or more of a variety ofrun-time environments. Thereafter, the virtual machine instance managerreceives user-initiated requests to execute code, and identifies apre-configured virtual machine instance to execute the code based onconfiguration information associated with the request. The virtualmachine instance manager can further allocate the identified virtualmachine instance to execute the user's code at least partly by creatingand configuring containers inside the allocated virtual machineinstance, and provisioning the containers with code of the task as wellas dependency code objects. Various embodiments for implementing avirtual machine instance manager and executing user code on virtualmachine instances is described in more detail in U.S. Pat. No.9,323,556, entitled “PROGRAMMATIC EVENT DETECTION AND MESSAGE GENERATIONFOR REQUESTS TO EXECUTE PROGRAM CODE” and filed Sep. 30, 2014 (“the '556Patent”), the entirety of which is hereby incorporated by reference.

As used herein, the term “virtual machine instance” is intended to referto an execution of software or other executable code that emulateshardware to provide an environment or platform on which software mayexecute (an “execution environment”). Virtual machine instances aregenerally executed by hardware devices, which may differ from thephysical hardware emulated by the virtual machine instance. For example,a virtual machine may emulate a first type of processor and memory whilebeing executed on a second type of processor and memory. Thus, virtualmachines can be utilized to execute software intended for a firstexecution environment (e.g., a first operating system) on a physicaldevice that is executing a second execution environment (e.g., a secondoperating system). In some instances, hardware emulated by a virtualmachine instance may be the same or similar to hardware of an underlyingdevice. For example, a device with a first type of processor mayimplement a plurality of virtual machine instances, each emulating aninstance of that first type of processor. Thus, virtual machineinstances can be used to divide a device into a number of logicalsub-devices (each referred to as a “virtual machine instance”). Whilevirtual machine instances can generally provide a level of abstractionaway from the hardware of an underlying physical device, thisabstraction is not required. For example, assume a device implements aplurality of virtual machine instances, each of which emulate hardwareidentical to that provided by the device. Under such a scenario, eachvirtual machine instance may allow a software application to executecode on the underlying hardware without translation, while maintaining alogical separation between software applications running on othervirtual machine instances. This process, which is generally referred toas “native execution,” may be utilized to increase the speed orperformance of virtual machine instances. Other techniques that allowdirect utilization of underlying hardware, such as hardware pass-throughtechniques, may be used, as well.

While a virtual machine executing an operating system is describedherein as one example of an execution environment, other executionenvironments are also possible. For example, tasks or other processesmay be executed within a software “container,” which provides a runtimeenvironment without itself providing virtualization of hardware.Containers may be implemented within virtual machines to provideadditional security, or may be run outside of a virtual machineinstance.

The foregoing aspects and many of the attendant advantages of thisdisclosure will become more readily appreciated as the same becomebetter understood by reference to the following description, when takenin conjunction with the accompanying drawings.

FIG. 1 is a block diagram of an illustrative operating environment 100in which an on-demand code execution system 110 may operate based oncommunication with user computing devices 102, auxiliary services 106,and network-based data storage services 108. By way of illustration,various example user computing devices 102 are shown in communicationwith the on-demand code execution system 110, including a desktopcomputer, laptop, and a mobile phone. In general, the user computingdevices 102 can be any computing device such as a desktop, laptop ortablet computer, personal computer, wearable computer, server, personaldigital assistant (PDA), hybrid PDA/mobile phone, mobile phone,electronic book reader, set-top box, voice command device, camera,digital media player, and the like. The on-demand code execution system110 may provide the user computing devices 102 with one or more userinterfaces, command-line interfaces (CLI), application programinginterfaces (API), and/or other programmatic interfaces for generatingand uploading user-executable source code (e.g., including metadataidentifying dependency code objects for the uploaded code), invoking theuser-provided source code (e.g., submitting a request to execute thesource code on the on-demand code execution system 110), schedulingevent-based jobs or timed jobs, tracking the user-provided source code,and/or viewing other logging or monitoring information related to theirrequests and/or source code. Although one or more embodiments may bedescribed herein as using a user interface, it should be appreciatedthat such embodiments may, additionally or alternatively, use any CLIs,APIs, or other programmatic interfaces.

The illustrative environment 100 further includes one or more auxiliaryservices 106, which can interact with the one-demand code executionenvironment 110 to implement desired functionality on behalf of a user.Auxiliary services 106 can correspond to network-connected computingdevices, such as servers, which generate data accessible to theone-demand code execution environment 110 or otherwise communicate tothe on-demand code execution environment 110. For example, the auxiliaryservices 106 can include web services (e.g., associated with the usercomputing devices 102, with the on-demand code execution system 110, orwith third parties), databases, really simple syndication (“RSS”)readers, social networking sites, or any other source ofnetwork-accessible service or data source. In some instances, auxiliaryservices 106 may be invoked by code execution on the on-demand codeexecution system 110, such as by API calls to the auxiliary services106. In some instances, auxiliary services 106 may be associated withthe on-demand code execution system 110, e.g., to provide billing orlogging services to the on-demand code execution system 110. In someinstances, auxiliary services 106 actively transmit information, such asAPI calls or other task-triggering information, to the on-demand codeexecution system 110. In other instances, auxiliary services 106 may bepassive, such that data is made available for access by the on-demandcode execution system 110. For example, components of the on-demand codeexecution system 110 may periodically poll such passive data sources,and trigger execution of tasks within the on-demand code executionsystem 110 based on the data provided. While depicted in FIG. 1 asdistinct from the user computing devices 102 and the on-demand codeexecution system 110, in some embodiments, various auxiliary services106 may be implemented by either the user computing devices 102 or theon-demand code execution system 110.

The illustrative environment 100 further includes one or morenetwork-based data storage services 108, configured to enable theon-demand code execution system 110 to store and retrieve data from oneor more persistent or substantially persistent data sources.Illustratively, the network-based data storage services 108 may enablethe on-demand code execution system 110 to store informationcorresponding to a task, such as source code or metadata, to storeadditional code objects representing dependencies of tasks, to retrievedata to be processed during execution of a task, and to storeinformation (e.g., results) regarding that execution. The network-baseddata storage services 108 may represent, for example, a relational ornon-relational database. In another example, the network-based datastorage services 108 may represent a network-attached storage (NAS),configured to provide access to data arranged as a file system. Thenetwork-based data storage services 108 may further enable the on-demandcode execution system 110 to query for and retrieve informationregarding data stored within the on-demand code execution system 110,such as by querying for a number of relevant files or records, sizes ofthose files or records, file or record names, file or record creationtimes, etc. In some instances, the network-based data storage services108 may provide additional functionality, such as the ability toseparate data into logical groups (e.g., groups associated withindividual accounts, etc.). While shown as distinct from the auxiliaryservices 106, the network-based data storage services 108 may in someinstances also represent a type of auxiliary service 106.

The user computing devices 102, auxiliary services 106, andnetwork-based data storage services 108 may communicate with theon-demand code execution system 110 via network 104, which may includeany wired network, wireless network, or combination thereof. Forexample, the network 104 may be a personal area network, local areanetwork, wide area network, over-the-air broadcast network (e.g., forradio or television), cable network, satellite network, cellulartelephone network, or combination thereof. As a further example, thenetwork 104 may be a publicly accessible network of linked networks,possibly operated by various distinct parties, such as the Internet. Insome embodiments, the network 104 may be a private or semi-privatenetwork, such as a corporate or university intranet. The network 104 mayinclude one or more wireless networks, such as a Global System forMobile Communications (GSM) network, a Code Division Multiple Access(CDMA) network, a Long Term Evolution (LTE) network, or any other typeof wireless network. The network 104 can use protocols and componentsfor communicating via the Internet or any of the other aforementionedtypes of networks. For example, the protocols used by the network 104may include Hypertext Transfer Protocol (HTTP), HTTP Secure (HTTPS),Message Queue Telemetry Transport (MQTT), Constrained ApplicationProtocol (CoAP), and the like. Protocols and components forcommunicating via the Internet or any of the other aforementioned typesof communication networks are well known to those skilled in the artand, thus, are not described in more detail herein.

The on-demand code execution system 110 is depicted in FIG. 1 asoperating in a distributed computing environment including severalcomputer systems that are interconnected using one or more computernetworks (not shown in FIG. 1). The on-demand code execution system 110could also operate within a computing environment having a fewer orgreater number of devices than are illustrated in FIG. 1. Thus, thedepiction of the on-demand code execution system 110 in FIG. 1 should betaken as illustrative and not limiting to the present disclosure. Forexample, the on-demand code execution system 110 or various constituentsthereof could implement various Web services components, hosted or“cloud” computing environments, and/or peer to peer networkconfigurations to implement at least a portion of the processesdescribed herein.

Further, the on-demand code execution system 110 may be implementeddirectly in hardware or software executed by hardware devices and may,for instance, include one or more physical or virtual serversimplemented on physical computer hardware configured to execute computerexecutable instructions for performing various features that will bedescribed herein. The one or more servers may be geographicallydispersed or geographically co-located, for instance, in one or moredata centers. In some instances, the one or more servers may operate aspart of a system of rapidly provisioned and released computingresources, often referred to as a “cloud computing environment.”

In the example of FIG. 1, the on-demand code execution system 110 isillustrated as connected to the network 104. In some embodiments, any ofthe components within the on-demand code execution system 110 cancommunicate with other components of the on-demand code execution system110 via the network 104. In other embodiments, not all components of theon-demand code execution system 110 are capable of communicating withother components of the virtual environment 100. In one example, onlythe frontend 120 (which may in some instances represent multiplefrontends 120) may be connected to the network 104, and other componentsof the on-demand code execution system 110 may communicate with othercomponents of the environment 100 via the frontends 120.

In FIG. 1, users, by way of user computing devices 102, may interactwith the on-demand code execution system 110 to provide source code, andestablish rules or logic defining when and how such code should beexecuted on the on-demand code execution system 110, thus establishing a“task.” For example, a user may wish to run a piece of code inconnection with a web or mobile application that the user has developed.One way of running the code would be to acquire virtual machineinstances from service providers who provide infrastructure as aservice, configure the virtual machine instances to suit the user'sneeds, and use the configured virtual machine instances to run the code.In order to avoid the complexity of this process, the user mayalternatively provide the code to the on-demand code execution system110, and request that the on-demand code execution system 110 executethe code using one or more pre-established virtual machine instances.The on-demand code execution system 110 can handle the acquisition andconfiguration of compute capacity (e.g., containers, instances, etc.,which are described in greater detail below) based on the code executionrequest, and execute the code using the compute capacity. The on-demandcode execution system 110 may automatically scale up and down based onthe volume, thereby relieving the user from the burden of having toworry about over-utilization (e.g., acquiring too little computingresources and suffering performance issues) or under-utilization (e.g.,acquiring more computing resources than necessary to run the codes, andthus overpaying).

To enable interaction with the on-demand code execution system 110, thesystem 110 includes one or more frontends 120, which enable interactionwith the on-demand code execution system 110. In an illustrativeembodiment, the frontends 120 serve as a “front door” to the otherservices provided by the on-demand code execution system 110, enablingusers (via user computing devices 102) to provide, request execution of,and view results of computer executable source code. The frontends 120include a variety of components to enable interaction between theon-demand code execution system 110 and other computing devices. Forexample, each frontend 120 may include a request interface 122 providinguser computing devices 102 with the ability to upload or otherwisecommunication user-specified code to the on-demand code execution system110 and to thereafter request execution of that code. In one embodiment,the request interface 122 communicates with external computing devices(e.g., user computing devices 102, auxiliary services 106, etc.) via agraphical user interface (GUI), CLI, or API. The frontends 120 processthe requests and makes sure that the requests are properly authorized.For example, the frontends 120 may determine whether the user associatedwith the request is authorized to access the source code specified inthe request.

References to source code as used herein may refer to any program code(e.g., a program, routine, subroutine, thread, etc.) written in aspecific program language. In the present disclosure, the terms “sourcecode,” “user code,” and “program code,” may be used interchangeably.Source code which has been compiled for execution on a specific deviceis generally referred to herein as “machine code.” Both “source code”and “machine code” are representations of the same instructions, whichmay be collectively referred to as “code.” Such code may be executed toachieve a specific function, for example, in connection with aparticular web application or mobile application developed by the user.As noted above, individual collections of code (e.g., to achieve aspecific function) are referred to herein as “tasks,” while specificexecutions of that code are referred to as “task executions” or simply“executions.” Source code for a task may be written, by way ofnon-limiting example, in JavaScript (e.g., nodejs), Java, Python, and/orRuby (and/or another programming language). Tasks may be “triggered” forexecution on the on-demand code execution system 110 in a variety ofmanners. In one embodiment, a user or other computing device maytransmit a request to execute a task may, which can generally bereferred to as “call” to execute of the task. Such calls may include thesource code (or the location thereof) to be executed and one or morearguments to be used for executing the source code. For example, a callmay provide the source code of a task along with the request to executethe task. In another example, a call may identify a previously uploadedtask by its name or an identifier. In yet another example, source codecorresponding to a task may be included in a call for the task, as wellas being uploaded in a separate location (e.g., storage of an auxiliaryservice 106 or a storage system internal to the on-demand code executionsystem 110) prior to the request being received by the on-demand codeexecution system 110. A request interface of the frontend 120 mayreceive calls to execute tasks as Hypertext Transfer Protocol Secure(HTTPS) requests from a user. Also, any information (e.g., headers andparameters) included in the HTTPS request may also be processed andutilized when executing a task. As discussed above, any other protocols,including, for example, HTTP, MQTT, and CoAP, may be used to transferthe message containing a task call to the request interface 122.

A call to execute a task may specify one or more third-party libraries(including native libraries) to be used along with the user codecorresponding to the task. In one embodiment, the call may provide tothe on-demand code execution system 110 a ZIP file containing the sourcecode and any libraries (and/or identifications of storage locationsthereof) corresponding to the task requested for execution. In someembodiments, the call includes metadata that indicates the source codeof the task to be executed, the language in which the source code iswritten, the user associated with the call, and/or the computingresources (e.g., memory, etc.) to be reserved for executing the sourcecode. For example, the source code of a task may be provided with thecall, previously uploaded by the user, provided by the on-demand codeexecution system 110 (e.g., standard routines), and/or provided by thirdparties. Illustratively, code not included within a call or previouslyuploaded by the user may be referenced within metadata of the task byuse of a URI associated with the code. In some embodiments, suchresource-level constraints (e.g., how much memory is to be allocated forexecuting a particular code) are specified for the particular task, andmay not vary over each execution of the task. In such cases, theon-demand code execution system 110 may have access to suchresource-level constraints before each individual call is received, andthe individual call may not specify such resource-level constraints. Insome embodiments, the call may specify other constraints such aspermission data that indicates what kind of permissions or authoritiesthat the call invokes to execute the task. Such permission data may beused by the on-demand code execution system 110 to access privateresources (e.g., on a private network). In some embodiments, individualcode sets may also be associated with permissions or authorizations. Forexample, a third party may submit a code object and designate the objectas readable by only a subset of users. The on-demand code executionsystem 110 may include functionality to enforce these permissions orauthorizations with respect to code sets.

In some embodiments, a call may specify the behavior that should beadopted for handling the call. In such embodiments, the call may includean indicator for enabling one or more execution modes in which toexecute the task referenced in the call. For example, the call mayinclude a flag or a header for indicating whether the task should beexecuted in a debug mode in which the debugging and/or logging outputthat may be generated in connection with the execution of the task isprovided back to the user (e.g., via a console user interface). In suchan example, the on-demand code execution system 110 may inspect the calland look for the flag or the header, and if it is present, the on-demandcode execution system 110 may modify the behavior (e.g., loggingfacilities) of the container in which the task is executed, and causethe output data to be provided back to the user. In some embodiments,the behavior/mode indicators are added to the call by the user interfaceprovided to the user by the on-demand code execution system 110. Otherfeatures such as source code profiling, remote debugging, etc. may alsobe enabled or disabled based on the indication provided in a call.

To manage requests for code execution, the frontend 120 can include anexecution queue (not shown in FIG. 1), which can maintain a record ofrequested task executions. Illustratively, the number of simultaneoustask executions by the on-demand code execution system 110 is limited,and as such, new task executions initiated at the on-demand codeexecution system 110 (e.g., via an API call, via a call from an executedor executing task, etc.) may be placed on the execution queue 124 andprocessed, e.g., in a first-in-first-out order. In some embodiments, theon-demand code execution system 110 may include multiple executionqueues, such as individual execution queues for each user account. Forexample, users of the on-demand code execution system 110 may desire tolimit the rate of task executions on the on-demand code execution system110 (e.g., for cost reasons). Thus, the on-demand code execution system110 may utilize an account-specific execution queue to throttle the rateof simultaneous task executions by a specific user account. In someinstances, the on-demand code execution system 110 may prioritize taskexecutions, such that task executions of specific accounts or ofspecified priorities bypass or are prioritized within the executionqueue. In other instances, the on-demand code execution system 110 mayexecute tasks immediately or substantially immediately after receiving acall for that task, and thus, the execution queue may be omitted.

As noted above, tasks may be triggered for execution at the on-demandcode execution system 110 based on explicit calls from user computingdevices 102 (e.g., as received at the request interface 122).Alternatively or additionally, tasks may be triggered for execution atthe on-demand code execution system 110 based on data retrieved from oneor more auxiliary services 106 or network-based data storage services108. To facilitate interaction with auxiliary services 106, the frontend120 can include a polling interface (not shown in FIG. 1), whichoperates to poll auxiliary services 106 or data storage services 108 fordata. Illustratively, the polling interface may periodically transmit arequest to one or more user-specified auxiliary services 106 or datastorage services 108 to retrieve any newly available data (e.g., socialnetwork “posts,” news articles, files, records, etc.), and to determinewhether that data corresponds to user-established criteria triggeringexecution a task on the on-demand code execution system 110.Illustratively, criteria for execution of a task may include, but is notlimited to, whether new data is available at the auxiliary services 106or data storage services 108, the type or content of the data, or timinginformation corresponding to the data. In some instances, the auxiliaryservices 106 or data storage services 108 may function to notify thefrontend 120 of the availability of new data, and thus the pollingservice may be unnecessary with respect to such services.

In addition to tasks executed based on explicit user calls and data fromauxiliary services 106, the on-demand code execution system 110 may insome instances operate to trigger execution of tasks independently. Forexample, the on-demand code execution system 110 may operate (based oninstructions from a user) to trigger execution of a task at each of anumber of specified time intervals (e.g., every 10 minutes).

The frontend 120 can further includes an output interface (not shown inFIG. 1) configured to output information regarding the execution oftasks on the on-demand code execution system 110. Illustratively, theoutput interface may transmit data regarding task executions (e.g.,results of a task, errors related to the task execution, or details ofthe task execution, such as total time required to complete theexecution, total data processed via the execution, etc.) to the usercomputing devices 102 or to auxiliary services 106, which may include,for example, billing or logging services. The output interface mayfurther enable transmission of data, such as service calls, to auxiliaryservices 106. For example, the output interface may be utilized duringexecution of a task to transmit an API request to an external service106 (e.g., to store data generated during execution of the task).

As shown in FIG. 1, in some embodiments, the on-demand code executionsystem 110 may include multiple frontends 120. In such embodiments, aload balancer (not shown in FIG. 1) may be provided to distribute theincoming calls to the multiple frontends 120, for example, in around-robin fashion. In some embodiments, the manner in which the loadbalancer distributes incoming calls to the multiple frontends 120 may bebased on the location or state of other components of the on-demand codeexecution system 110. For example, a load balancer may distribute callsto a geographically nearby frontend 120, or to a frontend with capacityto service the call. In instances where each frontend 120 corresponds toan individual instance of another component of the on-demand codeexecution system, such as the warming pools 130A or active pools 140Adescribed below, the load balancer may distribute calls according to thecapacities or loads on those other components. As will be described inmore detail below, calls may in some instances be distributed betweenfrontends 120 deterministically, such that a given call to execute atask will always (or almost always) be routed to the same frontend 120.This may, for example, assist in maintaining an accurate executionrecord for a task, to ensure that the task executes only a desirednumber of times. While distribution of calls via a load balancer isillustratively described, other distribution techniques, such as anycastrouting, will be apparent to those of skill in the art.

To execute tasks, the on-demand code execution system 110 includes oneor more warming pool managers 130, which “pre-warm” (e.g., initialize)virtual machine instances to enable tasks to be executed quickly,without the delay caused by initialization of the virtual machines. Theon-demand code execution system 110 further includes one or more workermanagers 140, which manage active virtual machine instances (e.g.,currently assigned to execute tasks in response to task calls).

The warming pool managers 130 ensure that virtual machine instances areready to be used by the worker managers 140 when the on-demand codeexecution system 110 detects an event triggering execution of a task onthe on-demand code execution system 110. In the example illustrated inFIG. 1, each warming pool manager 130 manages a corresponding warmingpool 130A, which is a group (sometimes referred to as a pool) ofpre-initialized and pre-configured virtual machine instances that may beused to execute tasks in response to triggering of those tasks. In someembodiments, the warming pool managers 130 cause virtual machineinstances to be booted up on one or more physical computing machineswithin the on-demand code execution system 110 and added to acorresponding warming pool 130A. For example, each warming pool manager130 may cause additional instances to be added to the correspondingwarming pool 130A based on the available capacity in the correspondingwarming pool 130A to service incoming calls. As will be described below,the warming pool managers 130 may further work in conjunction with othercomponents of the on-demand code execution system 110, such as theworker managers 140, to add or otherwise manage instances and/orcontainers in the warming pools 130A based on received pre-triggernotifications. In some embodiments, the warming pool managers 130 mayuse both physical computing devices within the on-demand code executionsystem 110 and one or more virtual machine instance services to acquireand maintain compute capacity that can be used to service calls receivedby the frontends 120. Further, the on-demand code execution system 110may comprise one or more logical knobs or switches for controlling(e.g., increasing or decreasing) the available capacity in the warmingpools 130A. For example, a system administrator may use such a knob orswitch to increase the capacity available (e.g., the number ofpre-booted instances) in the warming pools 130A during peak hours. Insome embodiments, virtual machine instances in the warming pools 130Acan be configured based on a predetermined set of configurationsindependent from a specific call to execute a task. The predeterminedset of configurations can correspond to various types of virtual machineinstances to execute tasks. The warming pool managers 130 can optimizetypes and numbers of virtual machine instances in the warming pools 130Abased on one or more metrics related to current or previous taskexecutions. Further, the warming pool managers 130 can establish ormodify the types and number of virtual machine instances in the warmingpools 130A based on pre-trigger notifications (e.g., by pre-initializingone or more virtual machine instances based on requirements of a taskexpected to be executed based on a received pre-trigger notification).

As shown in FIG. 1, instances may have operating systems (OS) and/orlanguage runtimes loaded thereon. For example, the warming pool 130Amanaged by a warming pool manager 130 can comprise instances 152, 154.The instance 152 includes an OS 152A and a runtime 152B. The instance154 includes an OS 154A. In some embodiments, the instances in thewarming pool 130A may also include containers (which may further containcopies of operating systems, runtimes, user codes, etc.), which aredescribed in greater detail below. Although the instance 152 is shown inFIG. 1 to include a single runtime, in other embodiments, the instancesdepicted in FIG. 1 may include two or more runtimes, each of which maybe used for running a different user code. In some embodiments, thewarming pool managers 130 may maintain a list of instances in acorresponding warming pool 130A. The list of instances may furtherspecify the configuration (e.g., OS, runtime, container, etc.) of theinstances.

In some embodiments, the virtual machine instances in a warming pool130A may be used to serve any user's calls. In one embodiment, all thevirtual machine instances in a warming pool 130A are configured in thesame or substantially similar manner. In another embodiment, the virtualmachine instances in a warming pool 130A may be configured differentlyto suit the needs of different users. For example, the virtual machineinstances may have different operating systems, different languageruntimes, and/or different libraries loaded thereon. In yet anotherembodiment, the virtual machine instances in a warming pool 130A may beconfigured in the same or substantially similar manner (e.g., with thesame OS, language runtimes, and/or libraries), but some of thoseinstances may have different container configurations. For example, oneinstance might have a container created therein for running code writtenin Python, and another instance might have a container created thereinfor running code written in Ruby.

The warming pool managers 130 may pre-configure the virtual machineinstances in a warming pool 130A, such that each virtual machineinstance is configured to satisfy at least one of the operatingconditions that may be requested or specified by a user when defining atask. In one embodiment, the operating conditions may include programlanguages in which the potential source code of a task may be written.For example, such languages may include Java, JavaScript, Python, Ruby,and the like. In some embodiments, the set of languages that the sourcecode of a task may be written in may be limited to a predetermined set(e.g., set of 4 languages, although in some embodiments sets of more orless than four languages are provided) in order to facilitatepre-initialization of the virtual machine instances that can satisfycalls to execute the task. For example, when the user is configuring atask via a user interface provided by the on-demand code executionsystem 110, the user interface may prompt the user to specify one of thepredetermined operating conditions for executing the task. In anotherexample, the service-level agreement (SLA) for utilizing the servicesprovided by the on-demand code execution system 110 may specify a set ofconditions (e.g., programming languages, computing resources, etc.) thattasks should satisfy, and the on-demand code execution system 110 mayassume that the tasks satisfy the set of conditions in handling therequests. In another example, operating conditions specified by a taskmay include: the amount of compute power to be used for executing thetask; the type of triggering event for a task (e.g., an API call, HTTPpacket transmission, detection of a specific data at an auxiliaryservice 106); the timeout for the task (e.g., threshold time after whichan execution of the task may be terminated); and security policies(e.g., may control which instances in the warming pools 130A are usableby which user), among other specified conditions.

One or more worker managers 140 manage the instances used for servicingincoming calls to execute tasks. In the example illustrated in FIG. 1,each worker managers 140 manages an active pool 140A, which is a group(sometimes referred to as a pool) of virtual machine instances,implemented by one or more physical host computing devices, that arecurrently assigned to one or more users. Although the virtual machineinstances are described here as being assigned to a particular user, insome embodiments, the instances may be assigned to a group of users,such that the instance is tied to the group of users and any member ofthe group can utilize resources on the instance. For example, the usersin the same group may belong to the same security group (e.g., based ontheir security credentials) such that executing one member's task in acontainer on a particular instance after another member's task has beenexecuted in another container on the same instance does not posesecurity risks. Similarly, the worker managers 140 may assign theinstances and the containers according to one or more policies thatdictate which requests can be executed in which containers and whichinstances can be assigned to which users. An example policy may specifythat instances are assigned to collections of users who share the sameaccount (e.g., account for accessing the services provided by theon-demand code execution system 110). In some embodiments, the requestsassociated with the same user group may share the same containers (e.g.,if the user codes associated therewith are identical). In someembodiments, a task does not differentiate between the different usersof the group and simply indicates the group to which the usersassociated with the task belong.

As shown in FIG. 1, instances may have operating systems (OS), languageruntimes, and containers. The containers may have individual copies ofthe OS, the runtimes, and user codes corresponding to various tasksloaded thereon. In the example of FIG. 1, the active pools 140A managedby a worker manager 140 includes the instances 156, 158. The instance156 has an OS 156A, runtimes 156B, 156C, and containers 156D, 156E. Thecontainer 156D includes a copy of the OS 156A, a copy of the runtime156B, and a copy of a code 156D-1. The container 156E includes a copy ofthe OS 156A, a copy of the runtime 156C, and a copy of a code 156E-1.The instance 158 has an OS 158A, runtimes 158B, 158C, 158E, 158F, acontainer 158D, and codes 158G, 158H. The container 158D has a copy ofthe OS 158A, a copy of the runtime 158B, and a copy of a code 158D-1. Asillustrated in FIG. 1, instances may have user codes loaded thereon, andcontainers within those instances may also have user codes loadedtherein. In some embodiments, the runtimes may also be user provided. Aruntime—sometimes referred to as a runtime system—generally includes aset of software designed to support execution of source code written ina given computer language. In accordance with the present disclosure,the runtime may include a JIT compiler configured to implement a varietyof trace-based optimization techniques. The runtime may further includean interpreter or other system to enable execution of the source codewithout or in cooperation with the JIT compiler. In some embodiments,the worker managers 140 may maintain a list of instances in an activepool 140A. The list of instances may further specify the configuration(e.g., OS, runtime, container, etc.) of the instances. In someembodiments, the worker managers 140 may have access to a list ofinstances in a warming pool 130A (e.g., including the number and type ofinstances). In other embodiments, the worker managers 140 requestscompute capacity from a warming pool manager 130 without havingknowledge of the virtual machine instances in a warming pool 130A.

In the example illustrated in FIG. 1, tasks are executed in isolatedexecution environments referred to as containers (e.g., containers 156D,156E, 158D). Containers are logical units created within a virtualmachine instance using the resources available on that instance. Forexample, each worker manager 140 may, based on information specified ina call to execute a task, create a new container or locate an existingcontainer in one of the instances in an active pool 140A and assigns thecontainer to the call to handle the execution of the task. In oneembodiment, such containers are implemented as Linux containers.

Once a triggering event to execute a task has been successfullyprocessed by a frontend 120, the frontend 120 passes a request to aworker manager 140 to execute the task. In one embodiment, each frontend120 may be associated with a corresponding worker manager 140 (e.g., aworker manager 140 co-located or geographically nearby to the frontend120) and thus, the frontend 120 may pass most or all requests to thatworker manager 140. In another embodiment, a frontend 120 may include alocation selector 126 configured to determine a worker manager 140 towhich to pass the execution request. Illustratively, to assist inimplementation of execution guarantees, the location selector 126 toselect the same worker manager 140 to receive each call to a task to thesame worker manager 140, such that the worker manager 140 can maintainan authoritative execution record for the task. In one embodiment, thelocation selector 126 may determine the worker manager 140 to receive acall based on hashing the call, and distributing the call to a workermanager 140 selected based on the hashed value (e.g., via a hash ring).Various other mechanisms for distributing calls between worker managers140 will be apparent to one of skill in the art.

On receiving a request to execute a task, a worker manager 140 findscapacity to execute a task on the on-demand code execution system 110.For example, if there exists a particular virtual machine instance inthe active pool 140A that has a container with the user code of the taskalready loaded therein (e.g., code 156D-1 shown in the container 156D),the worker manager 140 may assign the container to the task and causethe task to be executed in the container. Alternatively, if the usercode of the task is available in the local cache of one of the virtualmachine instances (e.g., codes 158G, 158H, which are stored on theinstance 158 but do not belong to any individual containers), the workermanager 140 may create a new container on such an instance, assign thecontainer to the task, and cause the user code of the task to be loadedand executed in the container.

If the worker manager 140 determines that the source code associatedwith the triggered task is not found on any of the instances (e.g.,either in a container or the local cache of an instance) in the activepool 140A, the worker manager 140 may determine whether any of theinstances in the active pool 140A is currently assigned to the userassociated with the triggered task and has compute capacity to handlethe triggered task. If there is such an instance, the worker manager 140may create a new container on the instance and assign the container toexecute the triggered task. Alternatively, the worker manager 140 mayfurther configure an existing container on the instance assigned to theuser, and assign the container to the triggered task. For example, theworker manager 140 may determine that the existing container may be usedto execute the task if a particular library demanded by the task isloaded thereon. In such a case, the worker manager 140 may load theparticular library and the code of the task onto the container and usethe container to execute the task.

If the active pool 140 does not contain any instances currently assignedto the user, the worker manager 140 pulls a new virtual machine instancefrom the warming pool 130A, assigns the instance to the user associatedwith the triggered task, creates a new container on the instance,assigns the container to the triggered task, and causes the source codeof the task to be downloaded and executed on the container.

In some embodiments, the on-demand code execution system 110 is adaptedto begin execution of a task shortly after it is received (e.g., by thefrontend 120). A time period can be determined as the difference in timebetween initiating execution of the task (e.g., in a container on avirtual machine instance associated with the user) and detecting anevent that triggers execution of the task (e.g., a call received by thefrontend 120). The on-demand code execution system 110 is adapted tobegin execution of a task within a time period that is less than apredetermined duration. In one embodiment, the predetermined duration is500 ms. In another embodiment, the predetermined duration is 300 ms. Inanother embodiment, the predetermined duration is 100 ms. In anotherembodiment, the predetermined duration is 50 ms. In another embodiment,the predetermined duration is 10 ms. In another embodiment, thepredetermined duration may be any value chosen from the range of 10 msto 500 ms. In some embodiments, the on-demand code execution system 110is adapted to begin execution of a task within a time period that isless than a predetermined duration if one or more conditions aresatisfied. For example, the one or more conditions may include any oneof: (1) the source code of the task is loaded on a container in theactive pool 140 at the time the request is received; (2) the source codeof the task is stored in the code cache of an instance in the activepool 140 at the time the call to the task is received; (3) the activepool 140A contains an instance assigned to the user associated with thecall at the time the call is received; or (4) the warming pool 130A hascapacity to handle the task at the time the event triggering executionof the task is detected.

Once the worker manager 140 locates one of the virtual machine instancesin the warming pool 130A that can be used to execute a task, the warmingpool manager 130 or the worker manger 140 takes the instance out of thewarming pool 130A and assigns it to the user associated with therequest. The assigned virtual machine instance is taken out of thewarming pool 130A and placed in the active pool 140A. In someembodiments, once the virtual machine instance has been assigned to aparticular user, the same virtual machine instance cannot be used toexecute tasks of any other user. This provides security benefits tousers by preventing possible co-mingling of user resources.Alternatively, in some embodiments, multiple containers belonging todifferent users (or assigned to requests associated with differentusers) may co-exist on a single virtual machine instance. Such anapproach may improve utilization of the available compute capacity.

In some embodiments, the on-demand code execution system 110 maymaintain a separate cache in which code of tasks are stored to serve asan intermediate level of caching system between the local cache of thevirtual machine instances and the account data store 164 (or othernetwork-based storage not shown in FIG. 1). The various scenarios thatthe worker manager 140 may encounter in servicing the call are describedin greater detail within the '556 Patent, incorporated by referenceabove (e.g., at FIG. 4 of the '556 Patent).

After the task has been executed, the worker manager 140 may tear downthe container used to execute the task to free up the resources itoccupied to be used for other containers in the instance. Alternatively,the worker manager 140 may keep the container running to use it toservice additional calls from the same user. For example, if anothercall associated with the same task that has already been loaded in thecontainer, the call can be assigned to the same container, therebyeliminating the delay associated with creating a new container andloading the code of the task in the container. In some embodiments, theworker manager 140 may tear down the instance in which the containerused to execute the task was created. Alternatively, the worker manager140 may keep the instance running to use it to service additional callsfrom the same user. The determination of whether to keep the containerand/or the instance running after the task is done executing may bebased on a threshold time, the type of the user, average task executionvolume of the user, and/or other operating conditions. For example,after a threshold time has passed (e.g., 5 minutes, 30 minutes, 1 hour,24 hours, 30 days, etc.) without any activity (e.g., task execution),the container and/or the virtual machine instance is shutdown (e.g.,deleted, terminated, etc.), and resources allocated thereto arereleased. In some embodiments, the threshold time passed before acontainer is torn down is shorter than the threshold time passed beforean instance is torn down.

In some embodiments, the on-demand code execution system 110 may providedata to one or more of the auxiliary services 106 as it executes tasksin response to triggering events. For example, the frontends 120 maycommunicate with the monitoring/logging/billing services included withinthe auxiliary services 106. The monitoring/logging/billing services mayinclude: a monitoring service for managing monitoring informationreceived from the on-demand code execution system 110, such as statusesof containers and instances on the on-demand code execution system 110;a logging service for managing logging information received from theon-demand code execution system 110, such as activities performed bycontainers and instances on the on-demand code execution system 110; anda billing service for generating billing information associated withexecuting user code on the on-demand code execution system 110 (e.g.,based on the monitoring information and/or the logging informationmanaged by the monitoring service and the logging service). In additionto the system-level activities that may be performed by themonitoring/logging/billing services (e.g., on behalf of the on-demandcode execution system 110), the monitoring/logging/billing services mayprovide application-level services on behalf of the tasks executed onthe on-demand code execution system 110. For example, themonitoring/logging/billing services may monitor and/or log variousinputs, outputs, or other data and parameters on behalf of the tasksbeing executed on the on-demand code execution system 110.

In some embodiments, the worker managers 140 may perform health checkson the instances and containers managed by the worker managers 140(e.g., those in a corresponding active pool 140A). For example, thehealth checks performed by a worker manager 140 may include determiningwhether the instances and the containers managed by the worker manager140 have any issues of (1) misconfigured networking and/or startupconfiguration, (2) exhausted memory, (3) corrupted file system, (4)incompatible kernel, and/or any other problems that may impair theperformance of the instances and the containers. In one embodiment, aworker manager 140 performs the health checks periodically (e.g., every5 minutes, every 30 minutes, every hour, every 24 hours, etc.). In someembodiments, the frequency of the health checks may be adjustedautomatically based on the result of the health checks. In otherembodiments, the frequency of the health checks may be adjusted based onuser requests. In some embodiments, a worker manager 140 may performsimilar health checks on the instances and/or containers in a warmingpool 130A. The instances and/or the containers in a warming pool 130Amay be managed either together with those instances and containers in anactive pool 140A or separately. In some embodiments, in the case wherethe health of the instances and/or the containers in a warming pool 130Ais managed separately from an active pool 140A, a warming pool manager130, instead of a worker manager 140, may perform the health checksdescribed above on the instances and/or the containers in a warming pool130A.

In the depicted example, virtual machine instances (“instances”) 152,154 are shown in a warming pool 130A managed by a warming pool manager130, and instances 156, 158 are shown in an active pool 140A managed bya worker manager 140. The illustration of the various components withinthe on-demand code execution system 110 is logical in nature and one ormore of the components can be implemented by a single computing deviceor multiple computing devices. For example, the instances 152, 154, 156,158 can be implemented on one or more physical computing devices indifferent various geographic regions. Similarly, each frontend 120,warming pool manager 130, and worker manager 140 can be implementedacross multiple physical computing devices. Alternatively, one or moreof a frontend 120, a warming pool manager 130, and a worker manager 140can be implemented on a single physical computing device. Although fourvirtual machine instances are shown in the example of FIG. 1, theembodiments described herein are not limited as such, and one skilled inthe art will appreciate that the on-demand code execution system 110 maycomprise any number of virtual machine instances implemented using anynumber of physical computing devices. Similarly, although multiplewarming pools 130A and active pools 140A are shown in the example ofFIG. 1, the embodiments described herein are not limited as such, andone skilled in the art will appreciate that the on-demand code executionsystem 110 may comprise any number of warming pools and active pools.

In accordance with embodiments of the present disclosure, the on-demandcode execution system 110 may further include an acceleration system 162configured to enable use of tracing information to assist in executionof source code across execution environments. Illustratively, theacceleration system 162 may collect tracing information generated duringexecution of source code within a first execution environment (e.g., aspecific virtual machine or container), and make that informationavailable to a second execution environment. In some instances, theacceleration system 162 may additionally or alternatively conduct AOTcompilation for source code based on tracing information related to apast executions, to generate machine code for the task. The accelerationsystem 162 may contain one or more compilers 162 to conduct suchexecution. Furthermore, the acceleration system 160 may in someinstances collect compiled machine code generated within a firstexecution environment (e.g., based on execution of a tracing JITcompiler within the first execution environment) and make such codeavailable to other execution environments, such that those otherenvironments may execute code of a task on the basis of the machinecode, rather than being required to themselves interpret or compilesource code for the task. Data include tracing information, machine codegenerated due to AOT compilation, or machine code generated duringexecution of a task in an execution environment (which data iscollectively referred to herein as “optimization information”) can bestored within an optimization information datastore 168. Theoptimization information datastore 168 can be implemented as any one ormore of a hard disk drive (HDD), solid state drive (SSDs) virtual diskdrive, tape drives network attached storage (NAS) device, or any otherpersistent or substantially persistent storage component. While shown aspart of the acceleration system 160, the acceleration system 160 mayadditionally or alternative implement the optimization informationdatastore on the data storage service 108.

While some functionalities are generally described herein with referenceto an individual component of the on-demand code execution system 110,other components or a combination of components may additionally oralternatively implement such functionalities. For example, a workermanager 140 may operate to facilitate sharing of optimizationinformation across execution environments, to conduct AOT compilation ofsource code based on tracing information gathered from an executionenvironment, etc., in a manner similar or identical to as describedherein with reference to an acceleration system 160.

FIG. 2 depicts a general architecture of a computing system (referencedas acceleration system 160) that operates to facilitatecross-environment application of tracing information on the on-demandcode execution system. The general architecture of the accelerationsystem 160 depicted in FIG. 2 includes an arrangement of computerhardware and software modules that may be used to implement aspects ofthe present disclosure. The hardware modules may be implemented withphysical electronic devices, as discussed in greater detail below. Theacceleration system 160 may include many more (or fewer) elements thanthose shown in FIG. 2. It is not necessary, however, that all of thesegenerally conventional elements be shown in order to provide an enablingdisclosure. Additionally, the general architecture illustrated in FIG. 2may be used to implement one or more of the other components illustratedin FIG. 1. As illustrated, the code analysis system 160 includes aprocessing unit 290, a network interface 292, a computer readable mediumdrive 294, and an input/output device interface 296, all of which maycommunicate with one another by way of a communication bus. The networkinterface 292 may provide connectivity to one or more networks orcomputing systems. The processing unit 290 may thus receive informationand instructions from other computing systems or services via thenetwork 104. The processing unit 290 may also communicate to and frommemory 280 and further provide output information for an optionaldisplay (not shown) via the input/output device interface 296. Theinput/output device interface 296 may also accept input from an optionalinput device (not shown).

The memory 280 may contain computer program instructions (grouped asmodules in some embodiments) that the processing unit 290 executes inorder to implement one or more aspects of the present disclosure. Thememory 280 generally includes random access memory (RAM), read onlymemory (ROM) and/or other persistent, auxiliary or non-transitorycomputer readable media. The memory 280 may store an operating system284 that provides computer program instructions for use by theprocessing unit 290 in the general administration and operation of thecode analysis system 160. The memory 280 may further include computerprogram instructions and other information for implementing aspects ofthe present disclosure. For example, in one embodiment, the memory 280includes a user interface unit 282 that generates user interfaces(and/or instructions therefor) for display upon a computing device,e.g., via a navigation and/or browsing interface such as a browser orapplication installed on the computing device. In addition, the memory280 may include and/or communicate with one or more data repositories(not shown), for example, to access user program codes and/or libraries.

In addition to and/or in combination with the user interface unit 282,the memory 280 may include a compiler 162 and management software 284that may be executed by the processing unit 290. In one embodiment, thecompiler 162 and management software 284 implement various aspects ofthe present disclosure. For example, the management software 284 mayinclude computer code executable to manage optimization information onthe on-demand code execution system 110 (e.g., tracking whichoptimization information is available generated for a given task). Thecompiler 162 may include code executable to conduct AOT compilation ofsource code based on tracing information related to execution of thesource code within an execution environment on the on-demand codeexecution system. In this regard, the compiler 162 may operate in amanner similar to a tracing JIT compiler, albeit in an AOT, rather thanJIT, fashion as well as on the basis of tracing information gatheredduring a past execution in a separate execution environment, rather thanon the basis of currently gathered tracing information (as in a tracingJIT compiler). In one embodiment, the compiler 162 may be a variation ofa tracing JIT compiler, modified to conduct AOT compilation of sourcecode based on tracing information related to a past execution of thecode in a different environment.

While the compiler 162 and management software 284 are shown in FIG. 2as part of the acceleration system 160, in other embodiments, all or aportion of the compiler 162 and management software 284 may beimplemented by other components of the on-demand code execution system110 and/or another computing device. For example, in certain embodimentsof the present disclosure, another computing device in communicationwith the on-demand code execution system 110 may include several modulesor components that operate similarly to the modules and componentsillustrated as part of the acceleration system 160.

In some embodiments, the acceleration system 160 may further includecomponents other than those illustrated in FIG. 2. For example, thememory 280 may further include an instance allocation unit forallocating execution environments to tasks, user code execution unit tofacilitate execution of tasks within the execution environments, or acontainer manager for managing creation, preparation, and configurationof containers within virtual machine instances.

With reference to FIG. 3, illustrative interactions are depicted inwhich the on-demand code execution system 110 generates traceinformation regarding execution of a task within a first executionenvironment 302 and stores the information at an acceleration system160. As discussed in more detail below, the trace information maythereafter be used to assist in execution of the task within otherexecution environments.

The interactions of FIG. 3 begin at (1), where a user device 102 submitsa request to the frontend 120 to execute the task. Submission of arequest may include transmission of specialized data to the frontend120, such as a HTTP packet or API call referencing the task. While theinteractions of FIG. 3 are described as including an explicit request toexecute the task by the user device 102, requests to execute the taskmay occur in a variety of manners, including submission of a call byauxiliary services 106 (not shown in FIG. 3A) or generation of a call bythe on-demand code execution system 110 (e.g., based on a rule to callthe alias when specific criteria are met, such as elapsing of a periodof time or detection of data on an auxiliary service 106). The requestmay include any information required to execute the task, such asparameters for execution, authentication information under which toexecute the task or to be used during execution of the task, etc.

At (2), frontend 120 instructs a worker manager 140 to execute the task.While not shown in FIG. 3, in some instances the frontend 120 mayperform additional operations prior to instructing a worker manager 140to execute a task, such as determining whether sufficient capacityexists to execute the task, queuing the task, determining accounts towhich to attribute execution of the task, etc. Such operations aredescribed in more detail in the '556 Patent.

At (4), the worker manager 140 then initiates execution of the taskwithin an execution environment. Execution of tasks is described in moredetail within the in the '556 Patent. However, in brief, the workermanager 140 may generate an execution environment (e.g., softwarecontainer or virtual machine instance) in which to execute the task,provision the execution environment with code required to execute thetask (e.g., source code of the task, a runtime system for that sourcecode, etc.), and initiate execution of the task within the environment.In accordance with embodiments of the present disclosure, the executionenvironment may be provisioned with a tracing JIT compiler whichoperates, during execution of the task, to trace that execution andgenerate tracing information therefrom.

Accordingly, at (5), the execution environment 302 begins execution ofthe task. Execution of the task may include, for example, interpretationor compilation of source code (which may include an intermediaterepresentation of the source code, such as bytecode) into machine codethat is executed within the environment 302. Additionally, at (6), theexecution environment (e.g., via operation of a tracing JIT compiler)generates trace data for the execution. Generally described, tracinginformation monitors an execution path within code, including detailssuch as loops which occur within the code, functions called in the code,the frequency of such loops or calls, and the like. Tracing mayadditionally include information such as type information for dataobjects whose type is determined at runtime (e.g., dynamically, asopposed to statically, typed objects). Collection of trace informationvia a tracing JIT compiler is known in the art, and thus will not bedescribed in detail herein. However, in contrast to traditionaloperation of a tracing JIT compiler, the tracing information, inaccordance with embodiments of the present disclosure, the may bepersisted between executions of code, and transmitted from the executionenvironment 302 for use in other execution environments.

Specifically, at (7), the execution environment transmits the trace datato the acceleration system 160. In one embodiment, interaction (7)occurs at the end of execution of the task within the executionenvironment. In another embodiment, interaction (7) occurs when theexecution environment 302 determines that sufficient tracing informationhas been collected to accurately represent execution of the code. Theamount of information to accurately represent code may be established,for example, by a designer of the tracing JIT compiler, or by anadministrator of the on-demand code execution system 110. The amount ofinformation may be specified, for example, as a size of the information(e.g., n bytes), a total runtime reflected in the tracing information(e.g., n seconds), an amount of loops monitored within the information(e.g., n loops), a combination thereof, etc.

As will be discussed below, the tracing information generated within theexecution environment 302 may thereafter be used to assist in executionof the task within other environments, such as by “jump starting”operation of a tracing JIT compiler within the other environments. Assuch, the tracing information may reduce or eliminate the typical JITwarmup period for subsequent environments. The acceleration system 160may therefore store the tracing information as optimization informationfor the task within the optimization information data store 168.

As an additional improvement, the acceleration system 160 mayadditionally or alternatively utilize the tracing information to conductAOT compilation for the task, in a manner that obtains many benefits oftraditional tracing-based JIT compilation within an AOT compilationcontext. Specifically, the acceleration system 160 may utilize thetracing information of the execution environment 302 to conduct AOTcompilation for source code of the task, while applying optimizationswhich traditionally are applied during JIT compilation (and which aretraditionally unavailable during AOT compilation). Such optimizationsinclude (but are not limited to) constant sub-expression elimination,dead-code elimination, register allocation, and loop-invariant codemotion. Each of these optimizations is known in the art, and thus willnot be described in detail herein. However, generally described,constant sub-expression elimination is an optimization in which a numberof instances of an identical expression within code are replaced by asingle variable holding a computed value (thus, e.g., reducingduplicative calculation but increasing the number of variables duringexecution). Dead-code elimination, in brief, is an optimization in whichcode known not to affect program results, such as unreachable code, isremoved (or “elided”). Register allocation, in brief, is the process ofoptimizing allocation of variables to registers of a central processingunit (“CPU”), as opposed to more slowly accessed memories of anenvironment. Loop-invariant code motion, in brief, is the process ofrelocating statements or expressions from within a loop of code tooutside that loop, when doing so does not affect program results (e.g.,the statements or expressions do not vary during implementation of theloop). Each of the above-noted optimizations may generally rely on or beimproved by tracing information for the task. For example, theabove-noted optimizations may have varying effect (and thus, actuallyincrease the speed or efficiency at which code executes) based on howcode executes in practice. Thus, by use of the tracing informationreceived from the execution environment 302, the acceleration system 160may determine which optimizations should be applied during AOTcompilation of the code. In some instances, the acceleration system 160may determine which portions of the source code should be compiled intomachine code in an AOT manner, as opposed to being left in their sourcecode form and interpreted during execution. For example, theacceleration system 160 may determine that a highly invariant portion ofcode (e.g., a common execution path) should be compiled into machinecode, while a highly variant portion of code (e.g., where the executionpath diverges) should be left uncompiled. The acceleration system 160may thereafter store the compiled machine code as optimizationinformation for the task within the optimization information data store168.

With reference to FIG. 4, additional or alternative interactions tothose of FIG. 3 for generating optimization information at the on-demandcode execution system 110 will be described. Specifically, while theinteractions of FIG. 3 relate to collection of tracing information froman execution environment 302, and to potentially conducting AOTcompilation for source code of a task based on the tracing information,the interactions of FIG. 4 relate to extraction from the executionenvironment 302 of machine code generated by a tracing JIT compileroperating within the execution environment 302. In traditionaloperation, a tracing JIT compiler operates during execution of sourcecode to generate machine code for portions of the source code. Duringgeneration of that machine code, a variety of optimizations (such asthose discussed above) may be applied to the compilation process.However, the machine code generated by a traditional tracing JITcompiler is generally not made available outside of the executionenvironment in which the tracing JIT compiler operates, and typically isnot maintained between executions of code. Thus, each time code isexecuted, the tracing JIT compiler must repeat it's “warm up” phase,recreating machine code from source code. This operation may beacceptable in environments with disparate execution environments, sincethe machine code generated by a tracing JIT compiler for a firstexecution environment may be unlikely to perform property when executedin a second execution environment. However, in the case of the on-demandcode execution system 110, the system 110 can be configured to generatenumerous environments with similar or identical architectures, such thatmachine code generated in a first execution environment will functionwhen executed in a second execution environment. Thus, in accordancewith embodiments of the present disclosure, it may be desirable toenable machine code generated by a tracing JIT compiler of a firstexecution environment to be extracted from that environment and storedfor later use in other execution environments.

The initial interactions of FIG. 4—specifically, interactions (1)through (6)—are similar to those described above with reference to FIG.3, and thus will not be re-described. However, in contrast to theinteractions depicted in FIG. 3, at interaction (7) of FIG. 4, theexecution environment 302 (e.g., via operation of a tracing JITcompiler) generates machine code from source code of the task, based onthe tracing information generated regarding the execution environment302. (While generation of machine code is not depicted as an interactionof FIG. 3, one skilled in the art will appreciate that a tracing JITcompiler within the execution environment 302 of FIG. 3 may neverthelessoperate to generate machine code in addition to the interactionsdepicted in FIG. 3. Thus, the interactions of FIGS. 3 and 4 may becomplimentary to one another, and in some instances occur concurrently.)During generation of machine code, the tracing JIT compiler may applyany number of optimizations based on tracing information, such as thosenoted above. Thereafter, at (8), the execution environment 302 transmitsthe machine code to the acceleration system 160. The acceleration system160 may thereafter store the compiled machine code as optimizationinformation for the task within the optimization information data store168, such that other execution environments executing the task mayutilize the machine code generated within the execution environment 302.

With reference to FIG. 5, illustrative interactions are depicted for theutilization of optimization information to enable efficient execution ofa task within an execution environment 502, based on optimizationinformation generated as a result of a prior execution in anotherexecution environment (e.g., the environment 302 of FIGS. 3 and 4). Likethe interactions of FIGS. 3 and 4, the interactions of FIG. 5 begin witha request to execute a task at the on-demand code execution system,which is passed through the frontend 120 and causes an instruction tothe worker manager 140 to execute the task, as shown in interactions (1)and (2) of FIG. 5. These interactions are similar to the correspondinginteractions of FIGS. 3 and 4 and thus will not be re-described.

The interactions of FIG. 5 continue at (3) where the worker manager 140detects that sufficient optimization information has been recorded toassist in execution of the task. While not shown in FIG. 5, detectingthat sufficient optimization information has been recorded may includecommunication with the acceleration system to determine whatoptimization information (if any) has been stored at that system 160.The amount of optimization information needed to assist in execution ofa task may be set, for example, by an administrator of the on-demandcode execution system 110, a developer of the runtime system in whichthe task will execute, a user who has submitted the task, or the like.The amount of information may be specified, for example, as a size ofthe information (e.g., n bytes), a total runtime reflected in theinformation (e.g., n seconds), an amount of loops monitored within theinformation (e.g., n loops), a combination thereof, etc. In oneembodiment, the existence of optimization information within theacceleration system 160 may be deemed sufficient. In some instances, theacceleration system 160 may be configured to store multiple types ofoptimization information for a given task, and the sufficiency ofinformation may depend at least in part on the type of informationstored. For example, the acceleration system 160 may store tracinginformation based at least partly on conditions under which the tracinginformation was collected (e.g., numbers, types, or values of parameterspassed to the task execution). The worker manager 140 may thus determinewhether sufficient optimization information is stored within theacceleration system 160 that matches or corresponds to conditions for acurrent execution (e.g., the same number of parameters, types ofparameters, parameter values, combinations thereof, etc.). As anotherexample, the acceleration system 160 may store machine code based atleast partly on the architecture (e.g., CPU type, operating system,etc.) for which the machine code was generated. The worker manager 140may thus determine whether sufficient optimization information is storedwithin the acceleration system 160 that matches or corresponds to anarchitecture of the execution environment 502 for a current execution.

After detecting that sufficient optimization information has beendetected, the interactions of FIG. 5 continue at (4), where the workermanager 140 initiates execution of the task in the execution environment502 using the optimization information. Accordingly, at (5), theexecution environment 502 retrieves the optimization information fromthe acceleration system 160. As noted above, the optimizationinformation may include, for example, tracing information related toprior execution of the task (e.g., within a different environment) ormachine code generated based on such tracing information (which machinecode may be generated by, for example, a tracing JIT compiler during theprior execution, or by an AOT compiler which applied optimizations basedon the tracing information). Thereafter, at (6), the executionenvironment executes the task using the optimization information.Illustratively, where the optimization information includes tracinginformation, execution of the task using the optimization informationmay include loading the tracing information by a tracing JIT compiler,in a manner similar to how a traditional tracing JIT compiler mayoperate after tracing information has been generated within a localenvironment. Thus, the tracing JIT compiler may be enabled to skip orgreatly reduce its own tracing step, enabling rapid optimization of JITcompiled code. Where the optimization information includes machine code,execution of the task using the optimization information may includedirect utilization of the machine code to execute the task, as opposedto use of source code for the task. Thus, use of machine code maygreatly reduce or even eliminate the need to conduct JIT compilation forthe task.

While the interactions of FIGS. 3-5 are described with respect to twoexecution environments (environments 302 and 502), one skilled in theart will appreciate that these interactions have been simplified forease of description. For example, in some embodiments, optimizationinformation generated in multiple execution environments may be utilizedto enable efficient execution of a task in another environment. Theoptimization information may be generated at different times,concurrently, or any combination thereof. The optimization informationmay be stored at the acceleration system for use by subsequentenvironments. In some instances, the acceleration system may aggregatethe optimization information from multiple environments. In otherinstances, the acceleration system may store the optimizationinformation for multiple environments separately, and a subsequentenvironment may retrieve multiple items of optimization information. Insome instances, a subsequent environment may retrieve and utilize a mostrecent set of optimization information, as the total amount ofoptimization information within the acceleration system 160 may be morethan is needed to efficiently execute the task. The amount of mostrecent optimization information may be equal, for example, to the amountof optimization information designated as sufficient to enable efficientexecution of the task.

As another variation on the interactions described above, in someinstances optimization information from a first version of a task may beutilized in accordance with the above-described interactions to enableefficient execution of a second version of a task. Versions of a taskmay correspond, for example, to alterations in source code for the taskover time (e.g., where a first version is the first set of code for thetask uploaded, a second version is a modification to that code, etc.).In some cases, the alterations between versions of a task may berelatively minor, and have little or no effect on optimizationinformation for the task. For example, a modification may be made to arelatively unused portion of code, and thus tracing information for anew version of the task may be expected to be similar or identical totracing information for an older version. Accordingly, in someembodiments, the on-demand code execution system 110 may be configuredto provision an execution environment for executing a new version of atask with optimization information generated by virtue of a pastexecution of a past version of the task. Because optimizationinformation generally includes “fallback” provisions, such thatexecution of code can continue even where the optimization informationis incorrect (albeit more slowly, as the source code for the task mustgenerally be interpreted for the functions for which the optimizationinformation failed), use of optimization information across versions ofthe same task would not be expected to inhibit execution of the newversion of the task.

With reference to FIG. 6, an illustrative routine 600 will be describedfor enabling cross-environment application of tracing information forimproved code execution within the on-demand code execution system 110.The routine 600 may be implemented, for example, by a worker manager 140of the system 110 in conjunction with the acceleration service 160.

The routine 600 begins at block 602, where a request to execute a taskon the system 110 is received. As noted above, the request may begenerated, for example, based on an API call received at the system 110.The amount of information required for sufficiency may be specified, forexample, as a size of the information (e.g., n bytes), a total runtimereflected in the tracing information (e.g., n seconds), an amount ofloops monitored within the information (e.g., n loops), a combinationthereof, etc. Implementation of block 604 may include communicationbetween the worker manager 140 and the acceleration system 160, forexample to determine what optimization information exists for the taskwithin the acceleration system 160. Implementation of block 604 mayfurther include determining which of the optimization information forthe task is applicable to the current execution. For example,implementation of block 604 may include determining whether theoptimization information within the system 160 was created based on atask execution under the same or similar parameters (e.g., the samenumber and/or type of parameters), based on task execution within thesame architecture, or the like.

Thereafter, implementation of the routine 600 varies according towhether sufficient optimization information exists. Where sufficientoptimization information does not exist, the routine 600 continues toblock 606, where execution occurs without the benefit of optimizationinformation (e.g., in a traditional manner). Alternatively, wheresufficient optimization information does exist, the routine 600continues to block 608, where an execution environment is provisionedwith the optimization information. Illustratively, tracing informationfrom one or more past task executions may be provisioned into theenvironment, such that a tracing JIT compiler can utilize the tracinginformation to conduct optimization JIT compilation for source code ofthe task. As another illustration, machine code generated for the task(e.g., by a tracing JIT compiler or by an AOT compiler operating withthe benefit of past tracing information) may be placed into theenvironment, such that the task can be executed at least partially viathe machine code, reducing or eliminating the need to compile orinterpret source code of the task. Thereafter, at block 610, executionof the code proceeds with the benefit of the optimization information.Thus, execution of the code is expected to proceed more quickly and/orefficiently, without requiring that the execution environment in whichthe code is executed itself generate the optimization information.

In either instance, the routine 600 then proceeds to block 612 where,during execution, tracing information is collected. This tracinginformation may then be used as or to generate additional optimizationinformation, enabling subsequent executions of the task to benefit fromthe additional optimization information. The tracing information may becollected, for example, via operation of a tracing JIT compiler withinthe execution environment for the task execution.

Optionally, at block 614, compilation of source code for the task may beconducted based on the available tracing information, which may includeboth tracing information collected for a current execution as well asother tracing information provisioned into the environment (e.g., asoptimization information generated by virtue of past task executions).In one embodiment, the compilation of block 614 is conducted by atracing JIT compiler operating in an execution environment for the task,and thus represents JIT compilation for source code of the task. Inanother embodiment, the compilation of block 614 is conducted by acompiler 162 of the acceleration service 160 as an AOT compilationprocess. In either instance, the compilation can result in machine codeusable to speed subsequent executions of the task.

At block 616, the optimization information generated by virtue of theroutine 600, which may include tracing information related to a currentexecution or machine code generated based on that tracing information(e.g., via JIT or AOT compilation), is stored at block 616. Thus,subsequent implementations of routine 600 can benefit from thisoptimization information.

All of the methods and processes described above may be embodied in, andfully automated via, software code modules executed by one or morecomputers or processors. The code modules may be stored in any type ofnon-transitory computer-readable medium or other computer storagedevice. Some or all of the methods may alternatively be embodied inspecialized computer hardware.

Conditional language such as, among others, “can,” “could,” “might” or“may,” unless specifically stated otherwise, are otherwise understoodwithin the context as used in general to present that certainembodiments include, while other embodiments do not include, certainfeatures, elements and/or steps. Thus, such conditional language is notgenerally intended to imply that features, elements and/or steps are inany way required for one or more embodiments or that one or moreembodiments necessarily include logic for deciding, with or without userinput or prompting, whether these features, elements and/or steps areincluded or are to be performed in any particular embodiment.

Disjunctive language such as the phrase “at least one of X, Y or Z,”unless specifically stated otherwise, is otherwise understood with thecontext as used in general to present that an item, term, etc., may beeither X, Y or Z, or any combination thereof (e.g., X, Y and/or Z).Thus, such disjunctive language is not generally intended to, and shouldnot, imply that certain embodiments require at least one of X, at leastone of Y or at least one of Z to each be present.

Unless otherwise explicitly stated, articles such as ‘a’ or ‘an’ shouldgenerally be interpreted to include one or more described items.Accordingly, phrases such as “a device configured to” are intended toinclude one or more recited devices. Such one or more recited devicescan also be collectively configured to carry out the stated recitations.For example, “a processor configured to carry out recitations A, B andC” can include a first processor configured to carry out recitation Aworking in conjunction with a second processor configured to carry outrecitations B and C.

Any routine descriptions, elements or blocks in the flow diagramsdescribed herein and/or depicted in the attached figures should beunderstood as potentially representing modules, segments, or portions ofcode which include one or more executable instructions for implementingspecific logical functions or elements in the routine. Alternateimplementations are included within the scope of the embodimentsdescribed herein in which elements or functions may be deleted, orexecuted out of order from that shown or discussed, includingsubstantially synchronously or in reverse order, depending on thefunctionality involved as would be understood by those skilled in theart.

It should be emphasized that many variations and modifications may bemade to the above-described embodiments, the elements of which are to beunderstood as being among other acceptable examples. All suchmodifications and variations are intended to be included herein withinthe scope of this disclosure and protected by the following claims.

1. A computer-implemented method comprising: initiating a firstexecution of source code within a first execution environment on anon-demand code execution system; tracing the first execution of thesource code to generate tracing information for the first execution, thetracing information reflecting at least an execution path for the firstexecution; using the tracing information for the first execution tocompile the source code into machine code optimized based at leastpartly on the tracing information for the first execution within thefirst execution environment; provisioning a second execution environmenton the on-demand code execution system with the machine code optimizedbased at least partly on the tracing information for the first executionwithin the first execution environment; and initiating a secondexecution of the source code within the second execution environment,wherein the second execution executes based at least partly on themachine code optimized based at least partly on the tracing informationfor the first execution within the first execution environment andwithout compiling at least a portion of the source code represented bythe machine code.
 2. The computer-implemented method of claim 1, whereinusing the tracing information for the first execution to compile thesource code into machine code optimized based at least partly on thetracing information for the first execution within the first executionenvironment comprises implementing a tracing just-in-time (JIT) compilerwithin the first execution environment.
 3. The computer-implementedmethod of claim 1, wherein using the tracing information for the firstexecution to compile the source code into machine code optimized basedat least partly on the tracing information for the first executionwithin the first execution environment comprises conducting a guidedahead-of-time (AOT) compilation of the source code.
 4. Thecomputer-implemented method of claim 3, wherein conducting the guidedAOT compilation of the source code comprises: transmitting the tracinginformation for the first execution from the first execution environmentto a third environment distinct from both the first and second executionenvironments; and conducting the guided AOT compilation in the thirdenvironment.
 5. The computer-implemented method of claim 3, whereinconducting the guided AOT compilation of the source code comprisesapplying at least one tracing-based optimization during the AOTcompilation, the at least one tracing-based optimization including atleast one of constant sub-expression elimination, dead-code elimination,register allocation, and loop-invariant code motion.
 6. Thecomputer-implemented method of claim 1 further comprising: provisioninga third execution environment on the on-demand code execution systemwith the machine code optimized based at least partly on the tracinginformation for the first execution within the first executionenvironment; and initiating a third execution of the source code withinthe third execution environment, wherein the third execution executesbased at least partly on the machine code optimized based at leastpartly on the tracing information for the first execution within thefirst execution environment and without compiling at least a portion ofthe source code represented by the machine code.
 7. A system comprising:one or more data stores storing computer-executable instructions; andone or more processors configured to execute the computer-executableinstructions, wherein the computer-executable instructions when executedcause the one or more processors to: initiate a first execution ofsource code within a first execution environment on an on-demand codeexecution system; trace the first execution of the source code togenerate tracing information for the first execution, the tracinginformation reflecting at least an execution path for the firstexecution; use the tracing information for the first execution tocompile the source code into machine code optimized based at leastpartly on the tracing information for the first execution within thefirst execution environment; provision a second execution environment onthe on-demand code execution system with the machine code optimizedbased at least partly on the tracing information for the first executionwithin the first execution environment; and initiate a second executionof the source code within the second execution environment, wherein thesecond execution executes based at least partly on the machine codeoptimized based at least partly on the tracing information for the firstexecution within the first execution environment and without compilingat least a portion of the source code represented by the machine code.8. The system of claim 7, wherein the computer-executable instructionswhen executed further cause the one or more processors to provision thesecond execution environment with the tracing information.
 9. The systemof claim 7, wherein the first execution environment corresponds to atleast one of a software container or a virtual machine instance.
 10. Thesystem of claim 7, wherein the computer-executable instructions whenexecuted further cause the one or more processors to generate the firstexecution environment in response to a request to execute the sourcecode and to deconstruct the first execution environment after the firstexecution completes.
 11. The system of claim 7, wherein the tracinginformation reflects an execution path during at least one priorexecution of the source code.
 12. The system of claim 7, wherein thecomputer-executable instructions when executed further cause the one ormore processors to, prior to provisioning the second executionenvironment on the on-demand code execution system with the machine codeoptimized based at least partly on the tracing information for the firstexecution within the first execution environment, to verify that anamount of the tracing information satisfies a sufficiency threshold. 13.One or more non-transitory computer-readable media comprisingcomputer-executable instructions that, when executed by a computingsystem, cause the computing system to: initiate a first execution ofsource code within a first execution environment on an on-demand codeexecution system; trace the first execution of the source code togenerate tracing information for the first execution, the tracinginformation reflecting at least an execution path for the firstexecution; use the tracing information for the first execution tocompile the source code into machine code optimized based at leastpartly on the tracing information for the first execution within thefirst execution environment; provision a second execution environment onthe on-demand code execution system with the machine code optimizedbased at least partly on the tracing information for the first executionwithin the first execution environment; and initiate a second executionof the source code within the second execution environment, wherein thesecond execution executes based at least partly on the machine codeoptimized based at least partly on the tracing information for the firstexecution within the first execution environment and without compilingat least a portion of the source code represented by the machine code.14. The one or more non-transitory computer-readable media of claim 13,wherein to use the tracing information for the first execution tocompile the source code into machine code optimized based at leastpartly on the tracing information for the first execution within thefirst execution environment, the computer-executable instructions causethe computing system to implement a tracing just-in-time (JIT) compilerwithin the first execution environment.
 15. The one or morenon-transitory computer-readable media of claim 14, wherein the tracingJIT compiler applies at least one tracing-based optimization during theAOT compilation, the at least one tracing-based optimization includingat least one of constant sub-expression elimination, dead-codeelimination, register allocation, and loop-invariant code motion. 16.The one or more non-transitory computer-readable media of claim 13,wherein the computer-executable instructions when executed further causethe computing system to provision the second execution environment withthe tracing information.
 17. The one or more non-transitorycomputer-readable media of claim 13, wherein the computer-executableinstructions when executed further cause the computing system toprovision the second execution environment with the machine codeoptimized in response to a request to execute the source code.
 18. Theone or more non-transitory computer-readable media of claim 13, whereinthe computer-executable instructions when executed further cause thecomputing system to, prior to provisioning the second executionenvironment on the on-demand code execution system with the machine codeoptimized based at least partly on the tracing information for the firstexecution within the first execution environment, to verify that anamount of the tracing information satisfies a sufficiency threshold. 19.The one or more non-transitory computer-readable media of claim 13,wherein to use the tracing information for the first execution tocompile the source code into machine code optimized based at leastpartly on the tracing information for the first execution within thefirst execution environment, the computer-executable instructions causethe computing system to conduct a guided ahead-of-time (AOT) compilationof the source code.
 20. The one or more non-transitory computer-readablemedia of claim 13, wherein to conduct the guided AOT compilation of thesource code, the computer-executable instructions cause the computingsystem to: transmit the tracing information for the first execution fromthe first execution environment to a third environment distinct fromboth the first and second execution environments; and conduct the guidedAOT compilation in the third environment.